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CHAPTER  1 
INTRODUCTION 


U.S.  Air  Force  responsibilities  require  that  field  units  can  perform  quick,  accurate 
assessments  of  pavement  conditions  at  remote  operating  locations.  A  standoff  assessment  is 
necessary  when  these  “quick-look”  evaluations  must  be  accomplished  in  areas  where  hostilities 
are  known  to  be  underway.  The  evaluation  system  should  provide  enough  information  to 
independently  assess  the  pavement  conditions  in  times  of  dire  need.  In  less  stringent 
environments,  data  produced  by  these  systems  would  complement  the  findings  of  the  combat 
control  teams  currently  in  use.  It  would  also  provide  a  non-destructive  pavement  analysis  system 
to  augment  peacetime  pavement  evaluation  techniques. 

The  study  is  confined  to  the  evaluation  of  electro-optical  systems  for  this  application. 
Both  active  (laser  source)  and  passive  (ambient  source)  systems  have  been  assessed.  The  sensors 
that  have  been  evaluated  are  limited  to  the  visible  (0.40-0.75  pm),  the  near  infrared  (IR)  (0.75- 
1.0  pm),  the  mid  IR  (1.0-6.0  pm),  and  the  far  IR  (6.0-12.0  pm).  The  mid  and  far  IR  are  in  the 
region  of  blackbody  or  thermal  radiation.  There  does  not  exist  a  single  detector  for  this  entire 
range  of  electro-magnetic  energy.  If  information  from  more  than  one  range  is  required,  then  it 
will  be  necessary  to  incorporate  multiple  detectors  in  the  system. 

In  Chapter  2,  the  various  types  of  detectors  are  discussed.  The  concepts  of  resolution, 
frame  rates,  field-of-view  (FOV),  and  scan  rates  for  remote  sensing  systems  are  introduced. 
Chapter  3  contains  a  detailed  description  of  the  passive  imaging  systems  as  applied  to  standoff 
pavement  evaluation.  The  active  imaging  systems  are  discussed  in  Chapter  4.  In  Chapter  5  the 
results  of  the  field  test  performed  at  Wright  Laboratory  in  September  of  1996  are  reported. 
Results  of  the  image  processing  on  the  data  collected  in  Chapter  5  are  covered  in  Chapter  6. 
Chapters  7  and  8  are  the  summaiy  and  recommendations  resulting  from  this  study. 
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CHAPTER  2 
BACKGROUND 


Many  advanced  optical  systems  have  been  developed  for  remote  sensing.  The  sensors  can 
be  classified  into  two  types:  staring  systems  or  scanning  systems.  The  detectors  employed  in  the 
staring  systems  are  most  commonly  Charge  Coupled  Device  (CCD)  cameras.  The  CCD  consists 
of  a  fixed,  two  dimensional  photodiode  (pixel)  array  for  any  given  camera.  The  CCD  camera  is 
employed  in  both  active  and  passive  remote  sensing.  The  scanning  systems  are  either  a  line  scan 
system  or  a  beam  fanning  system.  The  line  scan  system  has  been  applied  to  both  active  and 
passive  imaging  systems.  It  consists  of  a  single  detector,  such  as  a  photomultiplier  tube,  which 
scans  in  the  x-direction  while  the  motion  of  the  platform  yields  the  y-direction.  In  this  way  a  two 
dimensional  image  is  reconstructed.  A  beam  fanning  system  consists  of  producing  a  fan  beam 
from  a  laser  beam  by  expanding  the  beam  in  the  x-direction,  but  not  the  y-direction.  The  detector 
is  a  one  dimensional  photodiode  array  that  detects  the  fan  beam  in  the  x-direction.  The 
y-direction  is  obtained  from  the  forward  motion  of  the  platform.  The  two  dimensional  image  is 
reconstructed  from  the  sequential  series  of  data  stream  output  from  the  diode  array. 

In  all  cases  the  obtained  image  is  “pixelated”  or  digitized.  The  resolution  of  a  given  sensor 
system  will  depend  on  the  pixel  size,  the  focusing  optics  and  the  distance  from  the  target.  From 
Figure  2-1,  clearly 


tan(6 )  = 


_  Xeff 

2f  “  2L  ’ 


(2-1) 


where 


L  = 


h 

cos((p )  ’ 


(2-2) 


Xpixis  the  length  of  a  single  pixel,  0  is  the  half  angle  subtended  by  a  single  pixel  at  the  center  of  the 
focusing  lens,  his  the  height  of  the  sensor  above  the  ground,  cp  is  the  look  angle  of  the  sensor 
measured  from  vertical,  and  Xeffis  the  effective  length  subtended  by  a  single  pixel  at  the  surface  of 
the  pavement.  Combining  Equations  (2-1)  and  (2-2),  the  effective  length  is  then 


f  cos((p )  ' 


(2-3) 
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Figure  2-1.  Area  Subtended  by  a  Single  Pixel 


We  assume  that  Xeg  =  Xaouai » which  is  valid  for  small  look  angles.  For  a  square  pixel,  the 
effective  area  subtended  by  a  single  pixel  at  the  pavement  surface  is 


Aeff  A; 


pix 


^  h 
f  cos((p ) 


(2-4) 


If  the  look  angle  is  zero,  then  Equation  (2-4)  reduces  to 


A.,=AJy\  . 


(2-5) 


The  minimum  requirement  for  detection  of  a  feature  at  the  pavement  surface  is  that 
dimensions  of  the  feature  are  larger  than  the  area  subtended  by  a  single  pixel. 

If  one  dimension  of  the  feature  of  interest  is  the  same  size  as  x^,  then  the  length  of  its 
other  dimension  must  be  greater  than  the  length  subtended  by  a  pixel  in  order  to  detect  it.  A 
crack  in  the  pavement,  for  example,  would  be  detectable  if  its  width  was  equal  to  since  its 
length  would  extend  over  many  pixels.  A  small  hole  that  registers  in  only  one  pixel  would  not  be 
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detectable.  Typical  numbers  for  CCD  cameras  are:  10  |j,m  for  the  pixel  size,  and  16  mm  for  the 
focal  length  of  the  lens.  Values  for  Xeff  are  shown  as  functions  of  height  and  look  angle  in 
Table  2-1. 


Table  2-1.  Length  Subtended  by  a  Single  Pixel  for  Various  Heights  and  Look  Angles 


Look  Angle 

Height:  m  (ft) 

0“ 

10“ 

o 

o 

30 

(98.4) 

1.88  cm 

1.90  cm 

2.00  cm 

60 

(196.5) 

3.75  cm 

3.81  cm 

3.99  cm 

90 

(295.3) 

5.63  cm 

5.71cm 

5.99  cm 

120 

(393.7) 

7.50  cm 

7.62  cm 

7.98  cm 

The  absolute  value  of  the  magnification  of  the  system  is 


Thus,  the  resolution  can  be  improved  by  increasing /or  decreasing  h,  which  is  also  obvious  by 
inspection  of  Equation  (2-3).  However,  increasing /results  in  decreasing  the  FOV  of  the 
receiver,  which  may  not  be  an  option.  The  pixel  size  for  a  line  scan  system  is  determined  by  the 
scan  rate,  the  total  distance  scanned  in  the  object  plane,  and  the  velocity  of  the  platform.  All  of 
these  parameters  are  adjustable  and  will  be  discussed  in  greater  detail  in  Chapter  4. 

The  frame  rates  necessary  to  obtain  data  for  the  entire  area  of  interest,  i.e.,  the  airfield, 
will  be  determined  by  FOV  of  the  camera  system  and  the  velocity  of  the  platform.  The  FOV  is 
calculated  in  the  same  manner  as  the  area  subtended  by  a  single  pixel  except  that  the  area  of  the 
entire  image  at  the  focal  plane  is  used  instead  of  the  area  of  a  pixel.  For  a  CCD  camera,  this 
would  be  the  area  of  the  array.  For  a  512X512  pixel  CCD  array,  where  each  pixel  has  width 
10  |im,  the  focal  plane  has  a  width  of  0.51  cm.  Using  this  value  in  Equation  (2-3)  in  place  of  Xpu, 
the  length  subtended  by  the  CCD  array  at  the  pavement  surface  is  9.6  m  (31.5  ft)  at  a  height  of 
30  m  (98  ft)  and  zero  look  angle.  The  camera  FOV  for  these  parameters  is  9.6X9.6  m^.  The 
frame  rate  must  exceed  the  requirement  that  at  least  one  frame  is  captured  every  9.6  m.  If  the 
platform  velocity  is  180  kn  (121.6  m/s),  the  minimum  frame  rate  must  be  13  frames/s  in  order  to 
not  miss  any  of  the  pavement  area.  Standard  frame  rates  on  CCD  cameras  are  30  or  60  frames/s 
but  can  be  as  much  as  1000  frames/s.  The  FOV  could  be  decreased  by  ~2  orders  of  magnitude 
(thus  increasing  the  resolution)  and  imaging  the  length  of  the  airfield  is  still  possible  at  this 
platform  speed.  By  changing  the  FOV,  the  width  of  the  imaged  area  would  also  decrease,  which 
may  mean  multiple  passes  over  the  airfield  to  image  the  entire  strip.  For  a  scan  system,  the  scan 
rates  are  matched  with  the  platform  velocity  to  maintain  a  uniform  pixel  size  (width  equals 
length). 


It  is  evident  from  these  calculations  that  there  are  many  parameters  which  can  be  modified 
so  that  an  optimum  system  for  remote  airfield  evaluation  can  be  designed  based  on  requirements. 
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The  effect  of  increasing  or  decreasing  a  parameter  on  the  FOV,  resolution,  scan  rate,  and  frame 
rate  of  a  system  is  summarized  in  Table  2-2.  These  issues  are  relevant  to  all  systems  evaluated  in 
this  study.  The  salient  features  of  specific  systems  are  discussed  in  the  following  two  chapters. 


Table  2-2.  Summary  of  Variables  Determining  System  Parameters  for  Remote  Imaging 


1  Variable 

FOV 

Resolution 

Scan  Rate 

Frame  Rate 

decrease 

increase 

increase 

increase 

Increase  array  size 

increase 

no  effect 

NA 

decrease 

Increase  platform 
speed 

no  effect 

no  effect 

increase 

increase 

increase 

increase 

increase 

decrease 

Increase  viewing 
angle 

increase 

decrease 

increase 

decrease 

Decrease  focal  length 

increase 

decrease 

NA 

decrease 

2-4 


NSWCDD/TR-97/3 


CHAPTER  3 

PASSIVE  IMAGING  SYSTEMS 


Passive  imaging  systems  rely  on  ambient  background  illumination  from  the  sun.  They  are 
usually  easier  to  implement  and  are  generally  more  economical  than  active  imaging  systems.  The 
following  passive  imaging  systems  have  been  studied  and  evaluated  for  apphcation  to  remote 
airfield  assessment:  multispectral,  hyperspectral,  thermal,  stereoscopic,  and  polarimetry. 


3.1  MULTISPECTRAL 

Every  object  has  a  spectral  signature  associated  with  it.  When  illuminated  with  a 
broadband  source  (such  as  the  sun),  an  object  will  reflect  specific  wavelengths  and  absorb  others. 
The  first  is  its  reflection  spectra  and  the  second  is  its  absorption  spectra.  The  reflection  r ,  is  the 
fraction  of  the  total  incident  illumination  that  is  reflected  from  the  object,  while  the  absorption  a, 
is  the  fraction  of  the  incident  illumination  that  is  absorbed.  Assuming  all  the  incident  radiation  is 
either  reflected  or  absorbed  (transmission  is  zero),  by  conservation  of  energy,  r  +  a  =  1 . 

Multispectral  imaging  consists  of  examining  more  than  one  spectral  band  (usually  6-10)  of 
the  same  scene.  This  technique  is  used  extensively  in  mine  countermeasures.  If  the  spectral 
signature  of  a  certain  mine  is  known,  bandpass  filters  can  be  chosen  at  the  peaks  of  its  reflection 
spectra  so  that  the  mine  stands  out  in  the  scene.  Ideally,  the  background  reflection  spectra  (grass, 
sand,  etc.)  will  be  low  at  the  wavelengths  where  the  reflection  of  the  mine  is  high. 

In  most  cases,  the  spectral  signature  of  the  object  of  interest  is  not  known  in  advance.  The 
filters  are  chosen  at  wavelengths  where  the  background  reflection  spectra  are  low  and  non-natural 
objects  are  high.  The  coastal  battlefield  recoimaissance  analysis  (COBRA)^  system  uses  six 
narrow  bandpass  filters,  three  in  the  visible  and  three  in  the  near  IR.  These  are  mounted  on  a  filter 
wheel  and  the  wheel  is  attached  to  an  intensified  CCD  camera.  The  rotation  of  the  filter  wheel  is 
synchronized  with  the  frame  rate  of  the  camera  so  that  each  image  is  taken  with  a  different  filter 
for  six  frames.  Then  the  camera  is  focused  on  a  new  scene  and  six  more  images  are  acquired. 

The  benefits  are  twofold:  if  the  reflectivity  of  the  mine  (or  other  object  of  interest)  is  much  greater 
than  the  background,  there  is  a  large  increase  in  the  signal-to-noise  ratio  (SNR);  also  the  flashing 
of  the  images,  in  which  the  object  stands  out  in  some  but  not  in  others,  catches  the  attention  of  a 
human  observer  much  more  so  than  a  single  image  does. 

Except  for  distinguishing  between  material  types  (e.g.,  dirt  versus  concrete)  this  techitique 
is  not  amenable  to  the  application  of  remote  assessment  of  airstrips.  Firstly,  because  in  detecting 
cracks,  holes,  spalls,  etc.,  we  would  be  trying  to  distinguish  asphalt  from  asphalt  (or  concrete 
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from  concrete).  That  is,  a  hole  in  asphalt  will  have  the  same  spectral  signature  as  the  bulk  asphalt, 
therefore  it  would  be  indistinguishable.  Secondly,  assuming  we  could  distinguish  the  hole  from 
the  asphalt,  the  reflectance  spectra  of  both  asphalt  and  concrete  are  relatively  flat  in  the  visible 
(400-700  nm)  and  the  near  IR  (0.7- 1.0  pm).  These  spectra  are  reproduced  in  Figures  3-1  and  3-2 
(courtesy  of  J.  Salisbury,  Johns  Hopkins  University  (JHU)  spectral  library)  for  weathered  asphalt 
and  concrete  with  limestone  and  quartz  aggregate.  Since  there  are  no  peaks  or  special  features  in 
these  spectra  (at  these  wavelengths),  there  is  nothing  to  exploit. 


8 


GC 


weathered,  soiled  asphalt  w/some  limestone  and  quartz 


Wavelengtti  (microns) 


Figure  3-1.  Visible  and  Near  IR  Spectrum  of  Weathered  Asphalt 


3.2  HYPERSPECTRAL 

Hyperspectral  systems  are  more  t3q)ically  used  in  the  mid  to  far  IR.  The  difference 
between  hyperspectral  and  multispectral  is  the  number  of  spectral  bands  which  are  collected  for 
any  given  scene.  Hyperspectral  data  is  collected  with  a  scanning  interferometer.  The 
interferometer  t5q)es  vary  (difraction,  Michelson,  etc.)  but  the  physics  is  the  same.  Consider  a 
Michelson  type  interferometer  as  shown  in  Figure  3-3.  The  radiation  is  incident  at  the  input 
aperture  and  is  split  and  recombined  at  the  beam  splitter  BSl.  Mirror  Ml  is  fixed  while  mirror 
M2  is  scanned.  The  two-dimensional  CCD  collects  an  image  for  each  mirror  position  x.  This 
results  in  a  data  cube  for  each  scan  of  the  interferometer.  Each  frame  is  a  map  of  intensity  lix), 
for  a  particular  x.  The  frequency  spectrum  for  the  pixel  in  the  i***  row  and  the  j***  column  is 

=  (3-1) 
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soiled  gray  concrete  w/quartz  &  limestone  aggregate 


Wavelength  (raiCTons) 

Figure  3-2.  Visible  and  Near  IR  Spectrum  of  Weathered  Concrete 


By  using  k  =  27C  /  X  and  x  =  ct ,  the  analysis  can  be  done  in  frequency/time  space  rather  than 
wave  number/position.  The  sum  in  Equation  (3-1)  is  evaluated  for  each  k  at  each  pixel.  The  data 
is  then  sorted  according  to  wave  number  (or  frequency).  The  difference  in  wave  number  between 
bands  is 
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Ak  = 


1 

NAx' 


(3-2) 


Not  all  of  the  spectral  bands  contain  useful  information.  Typically,  some  knowledge  of 
which  bands  contain  useful  information  is  known  a  priori  so  that  data  can  be  quickly  accessed 
without  having  to  sort  through  every  spectral  band.  Once  the  useful  spectral  bands  have  been 
identified,  a  spectral  anomaly  detection  algorithm  can  be  applied  to  the  images.  The  approach  is 
to  examine  the  frames  for  localized  spectral  differences,  most  typically  implemented  by  comparing 
differences  between  a  pixel  of  interest  and  its  nearest  neighbors.  This  method  has  been  successful 
in  detecting  military  vehicles  in  cluttered  backgrounds.^ 

If  the  spectrum  of  the  object  of  interest  is  known  in  advance,  a  spectral  matched  filter 
algorithm  can  be  applied  to  the  data.  Another  approach,  which  does  not  require  a  priori 
knowledge  of  the  spectral  signature,  is  to  incorporate  a  spectral  reference  library  into  the  image 
processing  algorithm.  The  algorithm  then  matches  the  spectral  signatures  of  objects  in  the  scene 
with  the  library  contents  and  identifies  them  according  to  how  well  they  match.  The  algorithm 
also  identifies  objects  for  which  it  has  no  spectral  match.  This  method  is  very  useful  for  detecting 
camouflage  in  the  IR  where  the  spectral  reflectance  properties  result  from  the  type  of  material 
(non-natural  versus  plant  or  mineral)  rather  than  the  color  (as  in  the  visible).  A  combination  of 
these  two  methods,  along  with  the  thermal  differences  method  described  in  the  next  chapter,  has 
been  used  in  detecting  recently  buried  mines.^ 

This  method  is  not  particularly  well-suited  for  detecting  pavement  anomalies  because,  as 
in  the  multispectral  case,  the  cracks  and  holes  will  have  the  same  spectral  signature  as  the  bulk 
pavement.  However,  patches  in  a  runway  could  easily  be  detected  with  the  spectral  anomaly 
algorithm  provided  the  patch  material  is  different  than  the  bulk  material  (concrete  slurry  or  tar 
patches  on  asphalt,  for  example).  Also,  the  material  properties  of  the  bulk  pavement  can  be 
detected  using  this  method.  Silicon  dioxide  (Si02,  the  primary  component  of  quartz,  granite  and 
basalt),  for  example,  has  a  distinct  reststrahlen  (resonant  absorption)  doublet  feature  at  8-9  pm  in 
the  far  IR.  Calcium  carbonate  (the  primary  component  in  cement,  dolomite,  oyster  shells  and 
limestone)  has  a  distinct  reststrahlen  peak  at  1 1.2  pm.  These  minerals  also  have  distinct  features 
in  the  mid  IR.  One  could  use  a  spectral  library  of  the  basic  minerals  (silicon  dioxide,  calcium 
carbonate  etc.)  or  of  various  rocks  used  as  aggregate  (granite,  quartz,  basalt,  limestone  etc.).  The 
spectral  signature  of  the  pavement  would  then  be  compared  with  the  spectral  library  and  a  list  of 
components  and  percent  of  each  component  could  be  generated.  Figures  3-4  through  3-7 
(Courtesy  J.  Salisbury,  JHU)  show  the  mid  and  far  IR  spectra  of  asphalt  cement,  concrete  cement 
with  two  different  aggregates,  and  black  tar,  respectively.  It  is  easy  to  pick  out  the  quartz, 
carbonate  and  limestone  features  from  these  spectra.  Figure  3-7  is  featureless  because  tar  and 
asphalt  are  similar  to  ideal  blackbodies,  reflecting  very  little  and  absorbing  nearly  all  incident 
thermal  radiation. 

Hyperspectral  data  can  also  aid  in  assessing  dirt  landing  strips.  Disturbed  soil,  undisturbed 
soil  and  hard  packed  soil  are  easily  distinguishable  spectrally.  Thermal  images  very  clearly  show 
recent  tire  tracks  on  dirt  roads.  Disturbed  soil  has  a  rather  flat  spectrum  in  the  3-12  pm  region, 
while  the  spectrum  of  undisturbed  soil  has  the  feamres  associated  with  the  minerals  present  in  the 
soil.  Plants  and  weeds  grovraig  on  the  airstrip  or  in  cracks  or  holes  in  pavement  are  also  clearly 
distinguishable  spectrally.  Spectral  differences  in  soft  and  puffy  versus  hard  and  smooth  dry  lake 
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Rough  Asphalt  Cement  w/Aggregate  Showing 


Figure  3-4.  Reflectance  Spectrum  of  Rough,  Variegated  Asphalt  Cement  in  the  Mid-Far  IR 


Concrete  w/Quartz  Aggregate 


Figure  3-5.  Reflectance  Spectrum  of  Concrete  Cement  with  Quartz  Aggregate  in  the  Mid-Far  IR 
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Soiled  Concrete  w/Quartz  and  Limestone  Aggregate 


Figure  3-6.  Reflectance  Spectrum  of  Concrete  Cement  with  Quartz  and  Limestone  Aggregate 


Black,  Weathered  Tar 


Figure  3-7.  Reflectance  Spectrum  of  Black,  Weathered,  Matte  Tar 
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beds  has  been  exploited'*  for  determining  the  suitability  of  landing  the  space  shuttle.  The  soft  and 
puffy  lake  beds  contain  salts  which  push  up  the  dirt  and  make  it  fluffy.  The  salt  crystals  have 
bound  water  which  has  strong  absorption  peaks  at  1.66  |j,m  and  2.2  pm.  The  presence  of  this 
absorption  spectra  in  a  dry  lake  bed  in  the  desert  indicates  the  presence  of  these  salts,  thus 
determining  the  unsuitability  of  using  the  area  as  a  landing  strip. 


3.3  THERMAL 

Electromagnetic  radiation  in  the  mid  to  far  IR  (3-12  pm)  is  the  region  of  thermal 
blackbody  radiation.  The  radiant  exitance  of  a  blackbody  is  given  by 

„  27tc^h 

Mbb(?^.T)  = - ^ - ,  (3-3) 

A,^(e'^T  _i) 

where  c  is  the  speed  of  light,  h  is  Plank’s  constant,  k  is  Boltzman’s  constant,  T  is  the  absolute 
temperature  (in  degrees  Kelvin),  and  X  is  the  wavelength  of  the  radiation.  The  amount  of  thermal 
energy  an  object  will  radiate  depends  on  its  emissivity.  The  emissivity  is  a  material  property; 
certain  materials  will  absorb  and  re-emit  more  readily  than  others.  In  general,  a  black  painted 
object  will  absorb  and  re-emit  thermal  radiation  more  than  a  white  painted  object.  The  emissivity 
of  an  object  is  defined  relative  to  the  emissivity  of  an  ideal  blackbody  at  the  same  temperature  and 
wavelength,  i.e.. 


e  = 


M  object  (^>T) 


Mbb(X,T) 


(3-4) 


so  that  the  emissivity  of  an  ideal  blackbody  is  one.  If  the  transmittance  is  zero,  then  conservation 
of  energy  requires  that 


a  -I-  r  =  1  (3-5) 

where  a  is  the  absorption  and  r  is  the  reflectance.  In  thermal  equilibrium  £  =  a  (the  body  is 
absorbing  and  emitting  at  the  same  rate),  in  which  case  the  emissivity  can  be  written 

e  =  l-r.  (3-6) 

In  thermal  equilibrium  the  emissivity,  as  a  function  of  wavelength,  can  be  obtained  from  the 
reflection  spectra  of  the  various  materials  in  Figures  3-4  through  3-7.  For  black  asphalt  with  very 
little  aggregate,  the  emissivity  curve  is  plotted  in  Figure  3-8.  The  emissivity  for  weathered 
concrete  is  shown  in  Figure  3-9.  Both  asphalt  and  concrete  are  good  absorbers  (emitters)  in  the 
thermal  IR  region.  The  emissivity  of  the  black  asphalt  is  very  close  to  that  of  an  ideal  blackbody. 
The  emissivity  of  tar,  shown  in  Figure  3-10,  is  an  even  better  blackbody  radiator. 

Infrared  imaging  records  the  surface  temperature  in  the  scene.  The  surface  temperature  is 
dependent  on  the  subsurface,  surface  and  ambient  temperatures.  Heat  energy  is  transferred  by 
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Black  Asphalt  w/Little  Aggregate 


Figure  3-8.  Absorption  Spectrum  of  Black  Asphalt 


Soiled  Concrete  w/Quartz  and  Limestone  Aggregate 


Figure  3-9.  Absorption  Spectrum  for  Concrete 
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Black,  Weathered  Tar 


Figure  3-10.  Absorption  Spectrum  for  Black  Tar 


conduction,  convection  and  radiation.  The  ability  of  the  material  to  radiate  energy  is  measured  by 
the  emissivity,  which  is  strictly  a  surface  property.  In  general,  a  rough  surface  will  have  a  higher 
emissivity  than  smooth  surfaces.  Inspecting  large  areas  of  concrete  with  this  method  requires 
knowledge  of  surface  texture,  i.e.,  rubber  tire  tracks,  loose  sand  or  gravel,  etc.  Cracks,  holes, 
spalls  and  other  surface  anomalies  should  radiate  more  heat  than  the  surrounding  flat  surfaces 
because  there  is  more  surface  area  inside  a  crack  or  hole. 

Good  pavement  has  good  heat  conduction  and  negligible  heat  convection.  A  void  beneath 
the  surface  will  increase  the  insulation  of  the  pavement  by  reducing  the  heat  conduction  without 
increasing  the  convection.  The  dead  air  space  in  the  void  does  not  allow  for  convection  currents. 
In  a  thermal  image  the  region  over  the  void  will  appear  to  be  hotter  or  cooler  than  the 
surrounding  pavement. 

Thermal  imaging  has  been  successfully  used^’^  for  detecting  delaminations  and  voids  on 
bridges  and  airport  runways.  Successful  thermal  imaging  requires  a  temperature  difference 
between  the  material  under  inspection  and  the  ambient  background.  That  is,  if  the  pavement 
anomalies  are  in  thermal  equilibrium  with  the  background,  they  will  not  be  distinguishable  in  a 
thermal  image.  Therefore  a  heat  source  and  a  heat  sink,  with  temperature  flow,  is  required.  In 
the  daytime,  during  periods  of  heating,  the  heat  source  is  the  sun  and  the  pavement  is  the  heat 
sink.  At  night,  during  periods  of  cooling,  the  pavement  will  be  the  source  and  the  sky  will  be  the 
heat  sink.  The  two  to  ±ree  hours  starting  at  sunrise  is  the  best  time  for  thermal  imaging  during 
the  period  of  heating.  The  two  to  three  hours  starting  at  sunset  is  best  for  imaging  during  the 
period  of  cooling.  This  method  works  best  when  there  is  no  cloud  cover,  since  clouds  reflect  IR 
radiation.  High  gusts  of  wind  have  a  cooling  effect  on  the  surface,  therefore  testing  should  not  be 
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done  when  wind  speeds  are  high.  Finally,  testing  should  not  be  performed  when  there  is  standing 
water  on  the  surface.  Water  is  a  strong  absorber  in  the  IR  and  acts  like  a  mask. 


3.4  STEREOSCOPIC 

Stereoscopic  imaging  is  a  technique  that  employs  two  cameras  for  measuring  distances  to 
a  high  degree  of  accuracy.  Of  all  the  techniques  presented  in  this  chapter  (passive  imaging), 
stereoscopic  imaging  is  the  only  one  that  yields  information  about  the  surface  roughness  or  the 
depth  of  cracks,  holes,  spalls  and  trenches.  This  information  is  necessary  to  assign  a  pavement 
condition  index. 

Two  detectors  are  separated  by  known  distance  d,  as  shown  in  Figure  3-11,  and  the  look 
angles  0  and  (p  are  known.  When  a  point  on  the  surface  is  in  the  FOV  of  both  detectors,  the 
height  h,  between  the  detectors  and  that  point  can  be  calculated  in  the  following  way.  The  angle 
between  the  two  detectors  is  given  by 

C  =  180-e-(p.  (3-7) 


By  the  law  of  sines. 


A  = 


sintp 
sin^  ’ 


(3-8) 


and 


sin© 

h  =  A  sin  0  =  d  .  J  sin  0 .  (3-9) 

sinq 

The  accuracy  of  h  depends  on  the  accuracy’s  of  d,  0,  and  (p.  The  two  detectors  in  Figure  3-11 
can  be  synchronously  scanned  across  the  surface  to  obtain  the  heights  along  the  x-direction,  while 
the  forward  motion  of  the  platform  gives  the  heights  along  the  y-direction  for  subsequent  scans. 

A  three  dimensional  map  of  the  surface  can  then  be  reconstructed.  Alternatively,  two  CCD 
cameras  with  overlapping  FOVs  can  be  used.  The  angles  and  distances  between  pixels  must  then 
be  known  on  a  pixel-by-pixel  basis. 

This  method  is  particularly  useful  for  determining  the  severity  of  trenches  and  the  overall 
evenness  of  the  surface.  Stereoscopic  imaging  could  also  be  incorporated  into  a  thermal  scaimer 
to  determine  the  depth  of  subsurface  voids.  Further,  discrimination  between  surface  and 
subsurface  anomalies  in  thermal  images  can  be  achieved  with  this  technique. 


3.5  POLARIMETRY 

Radiation  from  the  sun  is  randomly  polarized.  When  randomly  polarized  radiation  is 
incident  on  a  surface,  certain  polarizations  will  be  preferentially  reflected  while  others  will  be 
preferentially  transmitted  or  absorbed  according  to  Fresnel’s  equations.  The  component  of 
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polarization  that  is  perpendicular  to  the  plane  of  incidence,  defined  in  Figure  3-12,  will  have  a 
higher  reflectance  than  the  parallel  component.  This  very  bright,  polarized  return  is  called 
specular  reflection.  It  is  responsible  for  the  glare  that  polarized  sunglasses  are  designed  to  reduce. 
Specular  reflection  from  the  sun  is  generally  polarized  in  the  horizontal  plane,  therefore  the  axis  of 
polarization  of  the  sunglasses  is  vertical  to  block  the  glare.  This  effect  is  exploited  in  imaging 
systems  by  mounting  a  polarizer,  with  the  polarization  axis  horizontal,  at  the  camera  input.  This 
will  enhance  detection  of  specular  reflections.  This  technique  is  particularly  useful  for  mine 
hunting  applications  because  metal  surfaces  generally  have  a  high  specular  return  whereas  the 
background  (earth,  sand,  vegetation)  does  not. 

For  application  to  remote  sensing  of  airstrips  this  method  may  aid  in  detecting  surface 
anomalies.  Any  sharp  edges  or  comers  may  induce  specular  reflections  depending  on  the  look 
angle  with  respect  to  the  angle  of  incidence  of  the  sunlight.  Most  pavements  do  not  induce 
specular  reflections  in  the  visible,  but  there  may  be  some  in  the  reflected  portion  of  the  thermal  IR. 
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Figure  3-12.  Geometry  for  Plane  of  Incidence  (k  is  the  momentum  vector  of  the  incident  radiation) 
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CHAPTER  4 

ACTIVE  IMAGING  SYSTEMS 


Active  imaging  systems  require  a  light  source,  most  typically  a  laser.  Incorporating  a  light 
source  into  the  imaging  system  increases  the  expense,  difficulty  of  implementation  and  operation 
and  should  not  be  considered  unless  there  is  a  compelling  reason  (significant  increase  in 
detectability).  Active  imaging  systems  are  ordinarily  employed  for  detecting  objects  embedded  in 
turbid  media  (moored  mines  in  ocean  water,  for  example).  Employing  a  laser  source  when 
imaging  through  turbid  media  significantly  improves  signal-to-noise  and  target  detectabiUty. 

Active  imaging  can  be  employed  at  any  wavelength,  ultraviolet  to  IR,  so  long  as  a  laser  or  light 
source  can  be  found  at  the  desired  wavelength.  This  discussion  is  therefore  applicable  to  aU  the 
thermal  and  visible  techniques  discussed  in  Chapter  3.  However,  the  multispectral  technique 
would  require  a  frequency  agile  laser  (able  to  switch  frequency  output  for  each  input  filter),  which 
does  not  currently  exist. 

For  the  purpose  of  remote  airfield  assessment.  Light  Detection  And  Ranging  (LDDAR) 
imaging  systems  are  not  necessary,  nor  are  they  recommended.  Except  in  conditions  of  poor 
visibility,  such  as  thick  fog  or  smoke,  passive  imaging  through  the  atmosphere  is  nearly  as 
effective.  This  is  true  for  LIDAR  systems  that  do  not  give  depth  information.  In  typical  LIDAR 
systems,  the  beam  is  expanded  and  a  two  dimensional  CCD  is  used  as  the  detector.  The  result  is  a 
two  dimensional  map  of  intensity  versus  x-y  positon.  Two  types  of  LIDAR  systems  which  do 
yield  depth  information  are  the  pulsed,  scanning  LIDAR  and  the  beam  fanning  LIDAR.  These 
systems  yield  information  about  the  surface  structure  that,  in  passive  systems,  only  the 
stereoscopic  technique  can  achieve  (Chapter  3.4).  The  details  for  the  scanning  and  the  beam 
fanning  LIDAR  are  presented  this  chapter. 


4.1  SCANNING  LIDAR 

The  line  scan  system  has  been  applied  to  both  active  and  passive  imaging  systems.  For 
active  systems,  either  a  continuous  or  a  pulsed  laser  is  used.  If  height  information  is  required,  a 
pulsed  laser  must  be  used.  The  narrow,  pulsed  beam  is  scanned  along  the  x-direction,  usually 
with  a  rotating  mirror  assembly.  A  second  mirror  in  the  scanning  assembly  collects  the  return 
signal  and  reflects  it  into  a  single  detector,  usually  a  photomultiplier  tube.  The  forward  motion  of 
the  platform  yields  the  y-axis  information  for  subsequent  scans.  In  this  way  a  two  dimensional 
image  can  be  constructed.  For  a  pulsed  laser  system,  the  height  information  is  obtained  by 
measuring  the  time  between  the  launch  of  the  laser  pulse  and  the  return  signal.  This  time  of  flight 
of  the  laser  pulse  corresponds  to  twice  the  distance  from  the  LIDAR  system  to  surface  of 
reflection;  i.e.,  2h  =  c  At ,  where  c  is  the  speed  of  tight  in  air.  The  height  information  can  be 
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obtained  on  a  pixel-by-pixel  basis  throughout  the  scan  and  a  three  dimensional  map  of  the  surface 
can  be  constructed. 

The  vertical  resolution  is  limited  by  the  rise  time  of  the  detector.  Current  state-of-the-art 
photomultiplier  tubes  have  rise  times  of  -0.5  ns.  This  corresponds  to  a  vertical  resolution  of 
15  cm  (or  0.5  ft).  State-of-the-art  avalanche  photodiodes  have  a  rise  time  of  -150  ps, 
corresponding  to  a  resolution  of  4.5  cm  (-  2  in.).  These  photodiodes  have  triple  the  resolution 
however  they  also  have  less  gain  and  dynamic  range  than  the  photomultipliers.  These  resolutions 
can  be  nominally  improved  by  leading  edge  detection  techniques.  One  researcher^  achieved  a 
resolution  of  2  mm  in  a  thermal  scanning  system  by  modulating  the  laser  source.  However, 
maintaining  the  millimeter  resolution  would  be  technically  difficult  if  the  system  were  to  be 
integrated  on  an  airborne  platform,  and  would  probably  result  in  imposing  serious  limits  on  the 
speed  and  height  of  the  platform. 

The  resolution  in  the  y-direction  (the  direction  of  motion  of  the  platform)  will  be 
determined  by  the  speed  of  the  platform  and  the  scan  rate.  Since  the  scan  rates  are  matched  with 
the  platform  velocity  to  maintain  a  uniform  pixel  size  (width  equals  length),  these  factors  will  also 
determine  the  resolution  in  the  scan  direction  (x-direction).  Typical  scan  rates  are  0.5-4.0  KHz. 
For  a  platform  moving  at  180  kn  (121.6  m/s)  and  a  scan  rate  of  4  KHz,  the  resolution  is  3  cm/hne 
in  the  y-direction.  If  the  readout  rate  is  set  at  1024  measurements/scan  (or  pixels/scan)  then  the 
FOV  in  the  x-direction  is  31  m  (102  ft).  This  requires  a  readout  rate  of  4  MHz,  which  is  within 
the  capability  of  a  photomultiplier  tube  in  a  continuous  line  scanner  (timing  cannot  be  measured), 
but  it  may  be  difficult  to  implement  for  the  range  requirements  of  the  pulsed  LIDAR  (timing 
measurements  for  height  information  on  each  readout).  Furthermore,  the  laser  pulse  repetition 
rate  must  be  matched  to  the  readout  rate  imposing  an  additional  requirement  on  the  system.  The 
pulse  repetition  rate  must  also  be  variable  to  allow  for  adjustments  in  readout  rates  based  on 
changes  in  platform  velocity. 

All  of  these  requirements  are  possible,  however  they  would  be  extremely  difficult  to 
implement  on  an  airborne  platform.  With  the  high  readout  rates  it  may  be  necessary  to  sacrifice 
intensity  information  in  each  pixel;  i.e.,  it  may  not  be  possible  to  measure  both  range  (by  timing) 
and  intensity  (integration  over  return  pulse)  at  high  speed.  A  change  in  platform  height  would  be 
translated  into  a  change  in  the  surface  structure  unless  a  correction  mechanism  is  built  in  to  the 
acquisition  system.  Furthermore,  all  parameters  must  be  continuously  adjustable.  For  resolutions 
of  0.5  ft,  it  hardly  seems  worthwhile. 


4.2  BEAM  FANNING 

Beam  fanning  is  an  active  imaging  technique  using  a  continuous  laser  source.  The  beam  is 
expanded  in  one  direction,  by  means  of  a  cylindrical  lens,  so  it  looks  like  a  fan.  The  detector  is  a 
one  dimensional  diode  array  (or  CCD).  This  system  does  not  require  scanning;  the  diode  array 
detects  the  laser  return  providing  the  x-axis  information  while  the  forward  motion  of  the  platform 
gives  the  y-axis  for  subsequent  readouts  of  the  array.  In  this  way  a  two  dimensional  image  of 
intensity  versus  x-y  position  can  be  constructed. 
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Used  in  this  way,  beam  fanning  does  not  provide  depth,  or  siuface  structure  information. 
However,  if  a  two  dimensional  CCD  is  used,  instead  of  a  linear  array,  and  there  is  a  separation  d, 
between  the  source  and  receiver,  then  surface  structure  can  be  obtained.  This  is  similar  to  the 
stereoscopic  method  outlined  in  Chapter  3.4  (compare  to  Figure  3-11).  The  configuration  for  this 
technique  is  shown  in  Figure  4- 1 .  The  surface  has  an  exaggerated  step  of  height  h2,  for 
illustrative  purposes.  If  the  surface  was  perfectly  flat,  h^  =  0 ,  and  the  camera  would  record  a 
straight  line  of  illuminated  pixels.  For  the  surface  shown  in  Figure  4-1,  the  camera  “sees”  a  step 
of  apparent  height  happ .  In  this  configuration  the  fan  beam  is  coincident  with  the  z-axis  so  that 
(p  =  90° .  Equation  (3-9)  then  reduces  to 


sinO 

h  =  d^— 7  =  dtanO .  (4-1) 

sin^ 

The  distance  d,  from  the  laser  source  to  the  row  of  pixels  illuminated  by  the  portion  of  the  line 
from  point  A  to  point  B,  along  with  the  look  angle  0,  are  used  in  Equation  (4-1)  to  calculate  hi. 
The  portion  of  the  line  from  point  C  to  point  D  illuminates  a  different  row  of  pixels.  The  distance 
from  the  laser  source  to  the  new  row  of  pixels  is  used  in  Equation  (4-1)  to  calculate  h  =  hj  -i-  h2 . 
Alternatively,  hi  can  be  calculated  from 

C  =  90-e,  (4-2) 

and 


where  happ  is  calculated  from  Equation  (2-3)  by  multiplying  Xpu  by  the  number  of  pixels 
illuminated  by  happ  at  the  CCD.  When  ^  =  0,  h^^p  =  0  also.  Thus,  when  the  source  and  receiver 

are  colinear  (d=0)  all  height  information  is  lost.  When  ^  =  90° ,  hj  =  h^pp .  As  in  the 

stereoscopic  imaging  case,  the  accuracy  (resolution)  with  which  h  can  be  measured  will  depend  on 
the  accuracy  that  the  source-receiver  separation  and  the  angles  are  known. 

In  this  configuration,  the  fan  beam  system  will  measure  longitudinal  non-uniformities  along 
the  direction  of  motion  of  the  platform.  The  frame  rate  of  the  CCD  would  have  to  be  very  fast 
since  only  one  line  at  a  time  is  actually  recorded  per  frame.  The  fastest  frame  rate  for  CCD 
cameras  is  about  1000  frames/s.  At  a  platform  velocity  of  121.6  m/s  (180  kn),  the  resolution  is 
12  cm/frame  (-5  in./frame). 

To  measure  non-uniformities  in  the  surface  transverse  to  the  direction  of  motion  a  second 
beam  would  have  to  be  incorporated  to  scan  in  the  x-direction.  This  involves  incorporating  scan 
rates  with  frame  rates  and  the  associated  difficulties  discussed  in  the  previous  chapter. 
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Figure  4-1 .  Diagram  of  Beam  Fanning  System  for  Measuring  Surface  Roughness 
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CHAPTER  5 
FIELD  TEST  RESULTS 


During  the  week  of  September  10, 1996,  a  field  test  was  performed  at  Wright  Laboratory, 
Tyndall  Air  Force  Base  (TAFB),  FL.  The  test  was  performed  by  Dr.  Nancy  Swanson  of  the 
Naval  Surface  Warfare  Center,  Dahlgren  Division  (NSWCDD),  Hugh  Welker  of  Applied 
Research  Associates,  and  Brad  Blume  of  Nichols  Research  Corporation.  The  purpose  of  the  test 
was  to  gather  images,  in  multiple  spectral  bands,  of  certain  areas  of  the  test  runway  at  Wright 
Laboratory  (visible  and  IR)  to  aid  in  the  study  of  the  application  to  remote  airfield  assessment 

There  were  two  cameras  used  in  this  test;  one  sensitive  in  the  visible  portion  of  the 
electromagnetic  spectrum  (400-700  run)  and  the  other  sensitive  in  the  IR  from  3-12  |im.  Areas  of 
the  test  runway  were  first  marked  out  in  grids.  The  areas  were  chosen  to  incorporate  a  variety  of 
pavement  conditions  to  assess  the  capability  of  detection  for  each  condition  in  each  spectral  band. 
The  areas  were  labeled  with  letters  from  A  to  M.  Each  area  was  measured  and  the  cracks,  faults, 
holes,  patched  craters  and  spalls,  and  voids  inside  the  area  were  measured  and  recorded.  Areas 
A-C  were  located  in  the  flexible  test  section  (sand  subgrade,  18  in.  sand  fiU,  12  in.  crushed 
Alabama  limestone  basecourse,  and  6  in.  asphalt)  with  the  gyratory  asphalt  mix.  Areas  D-G  were 
in  the  composite  test  section  (sand  subgrade,  6  in.  seashell,  10  in.  limestone  basecourse,  12  in. 
Portland  cement,  and  6  in.  asphalt)  with  the  gyratory  asphalt  mix.  Areas  H- J  were  in  the 
composite  test  section  with  the  Marshall  asphalt  mix.  Areas  K  and  L  were  in  the  flexible  test 
section  with  6  in.  of  Marshall  mix  asphalt.  Area  M  was  in  the  flexible  test  section  with  4  in.  of 
Marshall  mix  asphalt  AH  the  craters  were  patched  with  ASR-1  magnesium  phosphate  slimy  with 
no.  4  Alabama  crushed  limestone.  All  the  spalls  were  patched  with  just  the  magnesium  phosphate 
slurry. 


5.1  VISIBLE  IMAGES 

The  visible  camera  was  a  MTI 72  series  CCD  camera  with  768(H)X493(V)  active  pixel 
elements.  The  horizontal  dimension  of  the  pixels  was  1 1  tim  and  the  vertical  was  13  [am.  There 
were  two  lenses  used  throughout  the  test,  16  mm  and  12.5  mm.  The  height  of  the  camera  from 
the  surface  of  the  pavement  was  21.25  ft  (6.5  m).  The  viewing  angle  was  approximately  20°, 
though  this  varied  slightly  from  one  area  to  another.  The  viewing  angle  affects  only  the  vertical 
dimension  of  the  FOV.  TheFOVwas  11.3(H)X9.1(V)ft^forthe  16mmlensand 
14.4(H)X1 1.6(V)  for  the  12.5  mm  lens.  The  resolution  was  0.18  in.(4.5  mm)  in  the  horizontal 
direction  and  0.22  in.(5.6  mm)  in  the  vertical  direction  when  using  the  16  mm  lens.  For  the 
12.5  mm  lens,  the  resolution  was  0.23  in.(5.7  mm)  horizontal  and  0.28  in.(7.2  mm)  vertical. 
Areas  A-M  were  imaged  with  the  CCD  camera  for  seven  filters:  400, 450, 500, 550, 600, 650, 
and  700  nm,  aU  with  80  nm  bandwidth  except  for  the  550  nm  filter,  which  had  a  bandwidth  of 
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60  nm.  The  spectral  response  of  the  camera  (Figure  5-1)  peaks  at  580  nm.  We  found  that  the 
550  nm  filter  gave  the  sharpest,  clearest  image  while  the  400  and  700  nm  filters  yielded  the 
poorest  images.  This  is  obviously  due  to  the  spectral  response  of  the  camera  since  the  reflectivity 
spectra  of  asphalt  and  concrete  are  relatively  flat  in  the  region  from  400-700  nm,  as  shown  in 
Figures  3-1  and  3-2  respectively.  Concrete  has  a  higher  overall  reflectivity  than  asphalt  because  it 
is  much  lighter  in  color.  Figures  5-2  through  5-14  are  images  of  Areas  A-M  taken  with  the  CCD 
camera. 


Figure  5-1.  Spectral  Response  of  CCD  Camera 


Figure  5-2.  Visible  Image  of  Area  A:  a  7.5  ft  (W)  by  6  ft  (H)  grid.  Inside,  a  2.8  ft  by  2.8  ft  square  was  cut  from 
the  6  in.  thick  asphalt  and  lifted.  A  round,  2  ft  diameter,  6  in.  deep  hole  was  dug  under  the  asphalt  and  the  square 
was  replaced.  The  1/8  in.  cracks  and  the  1  in.  bolt  holes  were  filled  with  tar.  On  the  second  morning  of  the  test 
the  grid  was  painted  black.  This  image  was  taken  with  the  650  nm  filter  and  the  16  mm  lens. 
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Figure  5-4.  Visible  Image  of  Area  C:  a  10.2  (W)  by  10.2  (H)  ft  grid.  This  is  similar  to  Area  A  except  that  four 
2.8  ft  by  2.8  ft  squares  were  cut  and  lifted  out  and  a  6  in.  deep,  4  ft  diameter  hole  was  dug.  The  squares  were 
replaced  and  the  cracks  and  bolt  holes  were  filled  with  tar.  On  the  second  day  of  testing,  the  entire  area  was 
painted  black.  Filter:  650  nm;  lens:  16  mm. 
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Figure  5-5.  Visible  Image  of  Area  D:  8.5  ft  by  12.5  ft  grid.  There  is  a  large  crack,  approximately  2.5  ft  long  and 
0.5  to  1 .0  in.  wide,  just  under  the  “D”  at  the  upper  comer  of  the  grid.  The  other  cracks  were  cut  into  the  asphalt 
and  from  the  right  are:  3/8 ,  %,  5/4,  3 ,  and  5  1/8  in.  in  depth.  They  were  all  1/8  to  Va  in.  wide.  Filter:  650  nm; 
lens:  16  mm.  The  dark  spot  at  the  top  of  the  frame  is  water.  The  darker  areas  in  the  image  are  bare  pavement  and 
the  lighter  areas  are  ground  in  dirt. 


Figure  5-6.  Visible  Image  of  Area  E:  10.5  ft  (W)  by  12.5  ft  (H)  grid.  This  grid  is  entirely  inside  a  20  ft  diameter, 
18  in.  deep  patched  crater.  In  the  lower  left  comer  is  a  6.5  in.  diameter,  16.5  in.  deep  core  hole  with  1  ft  of 
standing  water.  There  is  a  ridge  intruding  in  the  upper  right  comer  that  is  1-1.5  in.  higher.  The  five  cracks  were 
cut  into  the  patch  material.  From  the  top  they  are  5/8,  9/8,  3,  5.5,  and  5/4  in.  deep  and  1/8-3/8  in.  wide.  The  core 
hole  to  the  left  was  not  inside  the  grid  but  was  6.5  in.  diameter  and  19  in.  deep.  Filter:  550  nm;  lens:  12.5  nun. 
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Figure  5-7.  Visible  Image  of  Area  F:  an  8.8  ft  (W)  by  10.3  ft  grid.  This  grid  has  a  5  ft  diameter,  18  in.  deep 
patched  crater  in  the  center.  The  entire  grid  has  been  painted  black.  There  is  a  6.5  in.  diameter,  17  in.  deep  core 
hole  in  the  center  of  the  crater.  There  are  numerous  hairline  cracks  and  uneven  areas  that  are  not  apparent  in  the 
visible  images  but  are  in  the  thermal.  Filter:  650  nm;  lens:  16  mm. 


Figure  5-8.  Visible  Image  of  Area  G:  a  5.7  ft  by  10.3  ft  grid.  This  grid  has  four  1.5  ft  diameter,  3-5  in.  deep 
patched  spalls.  The  upper  half  of  the  grid  has  been  painted  black.  In  the  center  of  the  second  patch  (from  the 
bottom)  is  a  6.5  in.  core  hole  with  the  6  in.,  cracked  core  in  the  hole.  There  is  a  3  in.  vertical  white  paint  strip  to 
the  left  of  the  patches  and  much  spilled  slurry.  Filter:  650  nm;  lens:  16  mm. 
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Figure  5-9.  Visible  Image  of  Area  H:  an  8.3  ft  (W)  by  8.8  ft  grid  with  a  5  ft  diameter,  18  in.  deep  patched  crater 
in  the  center.  At  6  in.  off  crater  center  is  a  6.5  in.  diameter,  16  in.  deep  core  hole  with  the  6  in.  core  left  in  the 
hole.  There  is  much  aggregate  showing  at  the  bottom  and  left  side  of  the  patch.  There  are  also  many  small  cracks 
in  the  upper  right  surface  of  the  patch.  Across  the  top  of  the  grid  is  a  1/8  to  Va  in.  wide,  1  in.  deep  crack.  There  is 
much  spilled  slurry  and  two  white,  painted  stripes.  Filter:  650  ran;  lens:  16  mm. 


Figure  5-10.  Visible  Image  of  Area  I:  a  14.8  ft  (W)  by  14.8  ft  grid  inside  a  20  ft  diameter,  18  in.  deep  patched 
crater.  The  6.5  in.  diameter  core  holes  are,  from  right  to  left,  20,  20,  and  17  in.  deep.  The  third  hole  has  5  in.  of 
standing  water.  The  first  hole  on  the  right  is  at  the  center  of  the  crater.  Not  shown  in  this  image,  at  the  upper  left 
edge  of  the  patch  is  another  core  hole  with  the  core  in  it.  In  the  lower  left  comer  is  a  large  area  of  slurry  that  is 
Va  in.  higher  than  the  rest  of  the  patch.  There  are  numerous  hairline  cracks  in  the  surface.  Filter:  650  ran;  lens: 
12.5  mm. 
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Figure  5-11.  Visible  Image  of  Area  J:  a  10.3  ft  (W)  by  9  ft  grid  with  three,  1 .5  ft  diameter,  3  to  5  in.  deep  patched 
spalls.  The  center  spall  has  a  6.5  in.  diameter  core  hole  filled  with  rocks  and  shells.  Not  visible  in  this  image,  in 
the  lower  left  comer  is  a  3.5  in.  diameter,  5.5  in.  deep  core  hole.  The  right  edge  of  the  grid  is  painted  gray,  the  left 
is  painted  black  with  three  strips  of  white  paint.  There  are  numerous  alligator  cracks  in  the  patches  and  spilled 
slurry.  Filter:  650  nm;  lens:  12.5  mm. 


Figure  5-12.  Visible  Image  of  Area  L:  a  9.4  ft  (W)  by  20  ft  rectangle  with  a  1 .2  ft  wide,  1 .5  in.  deep  rut  running 
the  length  of  the  grid.  At  the  left  edge  of  the  mt  toward  the  bottom  of  the  image  (lower  arrow)  is  a  1.5  in. 
diameter,  7  in.  deep  bolt  hole.  Toward  the  top  of  the  image,  left  of  the  mt  (upper  arrow)  is  another  1.5  in. 
diameter,  10  in.  deep  bolt  hole.  At  the  very  top  of  the  mt  is  a  weed  growing  in  the  center  of  the  mt.  There  is  dirt 
in  the  bottom  of  the  mt  2  ft  on  either  side  of  the  weed.  Just  above  and  to  the  left  of  the  weed  is  a  4  in.  diameter, 

Va  in.  deep  hole  and  to  the  left  of  that  is  a  1.5  in.  diameter,  11  in.  deep  bolt  hole.  These  last  two  are  outside  the  top 
of  the  frame.  At  the  top  right,  on  the  chalk  line,  is  another,  smaller  weed.  Filter:  650  nm;  lens:  16  mm. 
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Figure  5-13.  Visible  Image  of  Upper  and  Lower  Sections  of  Area  K.  The  entire  area  is  10.5  ft  (W)  by  20  ft  long. 
The  upper  image  was  taken  with  the  16  mm  lens  and  the  lower  with  the  12.5  mm  lens.  In  the  upper  left  comer  is  a 
5  ft  diameter,  18  in.  deep  crater  with  a  6  in.  core  in  a  6.5  in.  hole  at  the  center  of  the  crater.  The  left  side  of  the 
grid  is  painted  black.  There  are  numerous  old  paint  stripes  that  are  nearly  worn  off.  At  the  bottom  right  of  the 
grid  is  a  2  ft  wide  by  7.5  ft  long  concrete  patch.  The  white  spot  in  the  lower  frame  is  spilled  slurry.  Just  below  and 
to  the  left  of  the  “K”  is  a  1.5  in.  diameter  by  10  in.  deep  bolt  hole.  Filter:  650  nm. 
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Figure  5-14.  Visible  Image  of  Area  M:  a  13.5  ft  (across  trench)  by  11.5  ft  (parallel  to  trench)  grid.  The  main 
feature  is  a  5.6  ft  wide,  2.5-3.5  in.  deep  trench  carved  into  the  asphalt.  There  is  loose  gravel  and  asphalt  chunks  in 
the  trench.  On  the  far  side  of  the  trench  is  a  1 .2  ft  wide  by  2  in.  deep  rut.  There  is  hardened  slurry  in  the  bottom 
of  a  portion  of  the  rut  on  the  right  side. 


Intensity  values,  in  the  form  of  gray  scale  counts,  were  measured  for  various  features  in 
Area  G.  Results  are  graphed  in  Figure  5-15.  The  gray  scale  counts  were  measured  by  choosing 
an  area  in  the  image  containing  only  the  feature  of  interest  and  calculating  the  mean  and  standard 
deviation  of  the  pixels  inside  the  area. 
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Figure  5-15.  Normalized  Intensity,  Measured  in  Gray  Scale  Counts, 
vs.  Frequency  for  Various  Features  in  Area  G 
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The  lighter  concrete  had  higher  counts  than  the  darker  asphalt  and  black  paint,  as  expected 
from  the  magnitude  of  the  reflection  spectra  in  Figures  3-1  and  3-2.  The  shape  of  the  curves  in 
Figure  5-15  is  a  consequence  of  the  spectral  response  curve  of  the  camera  (Figure  5-1)  and  not  a 
change  in  reflectivity  of  the  surface  as  a  function  of  wavelength.  Therefore,  though  the  visible 
images  have  value,  there  is  no  added  information  to  be  gained  by  using  a  multispectral  approach, 
as  predicted  in  Chapter  3.1. 


5.2  THERMAL  IMAGES 

The  thermal  images  were  acquired  with  an  IR  scanning  radiometer.  Inframetrics  model 
760,  which  has  a  minimum  detectable  temperature  difference  of  +  0.1"C.  The  detector  in  this 
camera  is  a  mercury/cadmium/telluride  with  a  spectral  bandpass  of  3-12  pm.  There  are  two  filters 
that  can  be  rotated  into  the  path  of  the  radiation,  3-5  pm  and  8-12  pm.  The  detector  response  is 
poor  in  the  3-5  pm  range,  therefore  data  were  taken  only  for  the  entire  3-12  pm  range  (no  filter) 
and  with  the  8-12  pm  filter.  The  height  of  the  camera  from  the  surface  of  the  pavement  was 
21.25  ft  (6.5  m).  The  viewing  angle  was  approximately  20",  though  this  varied  slightly  from  one 
image  to  another.  The  average  distance  between  the  object  plane  and  the  image  plane  is 
L=h/cos(6)=22.6  ft  (6.9  m).  The  camera  must  be  used  with  a  telescope  for  proper  imaging.  We 
used  a  IX  telescope  and  a  3X.  The  3X  has  better  resolution  but  a  very  narrow  field  of  view. 

Using  the  IX  telescope,  the  instantaneous-field-of-view  (IFOV)  of  the  detector  is  1.8  mrad.  The 
minimum  resolution  is  Xmin=2L  tan(0.0018/2)=0.49  in.  (1.2  cm).  There  are  194  IFOVs/line,  thus 
the  horizontal  FOV  is  20".  This  corresponds  to  a  horizontal  FOV  of  7.9  ft  (2.4  m)  at  the  object 
plane.  The  vertical  FOV  is  15",  which  translates  to  6.0  ft  (1.8  m).  For  the  3X  telescope,  the 
minimum  resolution  is  0.16  in.  (4.0  mm),  the  horizontal  FOV  is  2.6  ft  (0.8  m),  and  the  vertical 
FOV  is  2.0  ft  (0.6  m).  It  is  important  to  realize  that  these  numbers  give  the  true  spatial  resolution 
even  though  the  data  is  digitized  at  256X256  pixels  per  image. 

The  most  interesting  data  were  taken  of  Areas  A  and  C,  where  the  voids  were  excavated 
beneath  6  in.  of  asphalt.  Since  air  is  a  poorer  conductor  of  heat  than  asphalt,  the  bulk  asphalt 
cools  at  a  more  rapid  rate  than  the  air  pocket.  The  air  void  is  hotter  than  the  bulk  material  and 
shows  up  as  a  light  spot  in  the  thermal  images.  Figures  5-16  through  5-18  are  thermal  images  of 
Area  A  taken  the  night  of  1 1  September.  Figures  5-19  through  5-21  are  thermal  images  of 
Area  C  also  taken  the  night  of  1 1  September.  The  cracks  and  bolt  holes  show  up  as  well  as  the 
subsurface  voids.  Figures  5-16  through  5-21  were  taken  with  the  IX  telescope.  Figures  5-22  and 
5-23  are  some  hot  spots  found  in  grid  J  taken  with  the  3X  telescope.  These  hot  spots  are 
imknown  subsurface  phenomena. 

Thermal  images  of  the  remaining  areas  are  shown  in  Figures  5-24  through  5-36.  Finally,  a 
grid  with  various  nuts,  bolts,  sand  and  aggregate  was  laid  out.  Figure  5-37  is  the  visible  image  of 
the  grid  with  the  objects  labeled.  Figure  5-38  is  a  thermal  image  of  the  grid.  The  objects  are 
clearly  visible  in  the  thermal  image,  showing  that  it  is  possible  to  also  distinguish  loose  debris  from 
bare  pavement  with  this  technique. 
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Figure  5-16.  Thermal  Image  of  Area  A:  taken  at  1701  hours;  ambient  temp:  34“C;  surface  temp:  42.5‘’C 
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Figure  5-17.  Thennal  Image  of  Area  A:  taken  at  1845  hours;  ambient  temp:  30°C;  surface  temp:  37.6‘’C 
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Figure  5-18.  Thermal  Image  of  Area  A:  taken  at  2030  hours;  ambient  temp:  28“C;  surface  temp:  35.2‘’C 


Figure  5-19.  Thermal  Image  of  Area  C:  taken  at  1722  hours;  ambient  temp:  32°C;  surface  temp:  42.5‘’C 
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Figure  5-20.  Thermal  Image  of  Area  C:  taken  at  1854  hours;  ambient  temp:  30“C;  surface  temp:  37.6°C 


Figure  5-21.  Thermal  Image  of  Area  C:  taken  at  2038  hours;  ambient  temp:  28®C;  surface  temp:  34.3‘’C 
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Figure  5-22.  Thermal  Image  of  Area  J:  taken  at  1524  hours;  ambient  temp:  35.5°C;  surface  temp:  43.0®C 


Figure  5-23 .  Thermal  Image  of  Area  J:  same  as  Figure  5-22  but  has  core  hole  at  far  right  for  reference 
(image  taken  facing  west) 
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Figure  5-24.  Thermal  Image  of  Area  B:  taken  at  1714  hours;  ambient  temp:  32.0°C;  surface  temp:  363°C;  temp 
inside  hole:  34.7  °C;  IX  telescope  lens 


Figure  5-25.  Thermal  Image  of  Area  D:  taken  at  1730  hours;  ambient  temp:  32.0‘’C;  surface  temp:  38.9°C: 
IX  telescope 
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Figure  5-26.  Thermal  Image  of  AreaE:  taken  at  1738  hours;  ambient  temp:  32.6®C;  surface  temp:  36.3'^C;  IX 
telescope 


Figure  5-27.  Thermal  Image  of  Area  F:  taken  at  1752  hours;  ambient  temp:  32.6°C;  surface  temp:  38.3°C; 
temp  inside  hole:  36.0  ^C;  IX  telescope 
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Figure  5-28.  Thermal  Image  of  AreaF:  taken  at  1032  hours;  ambient  temp:  34.0°C;  surface  temp:  40.5°C:  temp 
inside  hole:  36.4  °C:  3X  telescope.  There  was  standing  water  in  the  hole.  The  circled  area  shows  a  hot  spot 
indicating  a  region  of  delamination. 


Figure  5-29.  Thermal  Image  of  Upper  Portion  of  Area  G:  taken  at  1756  hours;  ambient  temp:  32.0'’C;  surface 
temp:  37.0°C;  IX  telescope 
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Figure  5-30.  Thermal  Image  of  Area  H:  taken  at  1928  hours;  ambient  temp:  29.0“C;  surface  temp;  33.6“C;  temp  of 
hole  crack:  34.5  °C;  IX  telescope 


Figure  5-31.  Thermal  Image  of  Area  I:  taken  at  1931  hours;  ambient  temp:  29SfO,  surface  temp:  33.6°C;  temp 
inside  hole  (lower  right):  33.9  °C;  IX  telescope.  Standing  water  in  lower  left  hole. 
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Figure  5-32.  Thermal  Image  of  Area  J:  taken  at  1937  hours;  ambient  temp;  29.0'’C;  surface  temp:  33.3‘’C;  dark 
spot  around  hole  is  water;  IX  telescope 


Figure  5-33.  Thermal  Image  of  Area  K:  taken  at  1820  hours;  ambient  temp:  30.5‘’C;  surface  temp:  35.2®C;  IX 
telescope.  Just  below  patch  is  an  old,  worn  paint  stripe. 
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Figure  5-34.  Thermal  Image  of  Area  L:  taken  at  1826  hours;  ambient  temp:  30.0°C;  surface  temp:  35.7°C;  IX 
telescope.  The  rut  runs  vertically  through  middle  of  image.  There  is  a  weed  growing  in  the  middle  of  the  trench. 
Just  to  the  right  of  that  is  a  4  in.  diameter  core  hole.  The  two  bright  spots  are  1 .5  in.  diameter  bolt  holes. 


Figure  5-35.  Thermal  Image  of  Upper  Left  Comer  of  Area  M:  taken  atl832  hours;  ambient  temp:  30.0°C;  surface 
temp:  35.5°C;  IX  telescope.  The  rut  mns  horizontally  across  the  image.  The  trench  is  visible  at  the  bottom  of  the 
image. 
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Figure  5-36.  Thermal  Image  of  Lower  Left  Comer  of  Area  M:  taken  at  2019  hours;  ambient  temp:  28.5"C;  surface 
temp:  33.1°C;  IX  telescope.  Loose  asphalt  is  visible  in  the  comer  of  the  trench. 
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Figure  5-38.  Thermal  Image  of  Grid  in  Figure  5-37:  taken  at  1826  hours;  ambient  temp:  30.0  “C.  The  edge  of  the 
pavement  is  clearly  visible  along  with  the  narrow  paint  stripe. 


The  thermal  images  clearly  show  both  surface  and  subsurface  effects.  The  surface  effects 
show  up  more  clearly  in  the  early  morning  during  periods  of  heating.  The  subsurface  voids  show 
more  clearly  in  the  late  evening  during  periods  of  cooling.  Contrast  ratios  were  calculated  for  the 
voids  in  Areas  A  and  C,  and  for  the  cracks  in  Areas  D  and  E.  The  contrast  ratio  is  defined  as 

>-i _  ^max  ^min  /c 

~  I  +I  .  ’  ^  ^ 

max  mm 

where  Imax  and  Imin  are  the  maximum  and  minimum  pixel  gray  scale  values.  An  area  was  chosen 
containing  only  pixels  illuminated  by  the  crack  or  void  and  the  mean  and  standard  deviation  of  the 
gray  scale  value  (0-255)  of  those  pixels  was  calculated.  The  mean  was  assigned  to  Imax  ■  A 
second  area  was  chosen  on  the  background  near  the  crack  or  void  and  the  process  was  repeated. 
The  mean  value  of  the  background  was  assigned  to  The  contrast  was  then  calculated  using 
Equation  (5-1).  This  process  was  repeated  for  several  frames  of  the  same  image  taken  at  the 
same  time.  The  contrast  ratio  for  the  voids  is  plotted  versus  the  time  of  day  in  Figure  5-39.  The 
negative  values  for  the  contrast  ratios  mean  that  the  void  (or  crack)  was  cooler  than  the 
background  and  was  therefore  darker  (less  counts)  in  the  image.  The  contrast  ratios  for  the 
cracks  in  Areas  D  and  E  versus  time  of  day  are  plotted  in  Figures  5-40  and  5-41.  At  1000  hours, 
the  cracks  in  the  concrete  slurry  of  Area  E  were  cooler  than  the  bulk  material  resulting  in  a 
negative  contrast  ratio.  Figure  5-42  is  the  contrast  ratio  versus  time  of  day  for  Area  E  with  the 
vertical  axis  adjusted  to  suppress  the  negative  contrast  values.  This  was  done  for  easier 
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.  Contrast  Ratio  vs.  Time  of  Day  for  Voids  in  Areas  A  and  C 
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Figure  5*40.  Contrast  Ratio  vs.  Time  of  Day  for  Cracks  in  Area  D 
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Figure  5-41.  Contrast  Ratio  vs.  Time  of  Day  for  Cracks  in  Area  E 
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Figure  5-42.  Contrast  Ratio  vs.  Time  of  Day  for  Cracks  in  Area  E  with  Negative  Values  Suppressed 
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comparison  to  Figure  5-40,  the  cracks  in  Area  D.  The  statistical  error  was  first  calculated  from 
the  standard  deviations  of  the  pixel  counts,  but  it  was  found  to  be  much  smaller  than  the  standard 
deviation  of  the  frame-to-frame  contrast  ratios.  The  frame-to-frame  standard  deviation  was 
therefore  used  as  error  bars  in  the  plots. 

The  cracks  were  cut  into  the  pavement  at  varying  depths.  They  are  numbered  from  one  to 
five  starting  with  the  first  crack  at  the  bottom  of  the  frames  in  Figures  5-25  and  5-26.  The  depth 
of  each  crack  is  shown  in  the  legend  on  each  plot  for  Areas  D  and  E. 

Temperature  measurements  were  taken  at  the  surface  and  inside  each  crack  during  each 
measurement.  Plots  of  contrast  ratios  versus  temperature  are  shown  in  Figures  5-43  through  5-46 
for  the  voids  and  the  cracks  in  Areas  D  and  E. 

The  voids  are  cooler  than  the  bulk  material  in  the  early  morning  during  periods  of  heating 
and  the  contrast  is  poor.  The  contrast  ratios  measured  in  the  evening  are  lower  than  they  should 
be  because  Areas  A  and  C  were  painted  black  that  morning.  The  black  paint  absorbed  and 
retained  the  heat  more  than  the  bare  pavement,  resulting  in  a  higher  background  count.  The  plot 
in  Figure  5-43  of  contrast  ratio  versus  temperature  shows  that  the  voids  are  more  visible  when  the 
temperature  is  maximum.  This  may  be  misleading,  since  the  maximum  temperature  measurement 
corresponds  to  that  taken  at  1400  hours.  This  measurement  was  taken  on  the  day  before  the 
areas  were  painted  black.  Painting  these  areas  has,  unfortunately,  skewed  this  data. 

The  contrast  ratios  for  the  cracks  are  greatest  in  the  early  morning  during  periods  of 
heating.  This  was  also  true  for  all  the  surface  phenomena.  There  is  a  crossover  point  about 
midmoming  when  the  temperature  of  the  bulk  material  exceeds  the  temperature  of  the  surface 
anomaly.  When  this  crossover  occms  will  depend  on  the  thermal  properties  of  the  material;  the 
thermal  conductivity  and  the  specific  heat.  While  these  quantities  are  independent  of  each  other, 
they  both  contribute  to  the  apparent  temperature  of  the  surface  as  measured  by  the  thermal 
camera.  The  thermal  conductivity  is  a  measure  of  the  rate  that  energy  is  transferred  through  the 
bulk  material.  If  the  conductivity  is  high,  the  energy  wiU  be  rapidly  distributed  throughout  the 
bulk  of  the  material.  The  surface  temperature  will  appear  to  be  less  than  that  of  a  material  with 
lower  conductivity  because,  for  the  lower  conductivity  material,  the  heat  energy  will  remain  at  the 
surface  longer.  From  the  CRC  Handbook,  the  thermal  conductivity  of  asphalt  is  0.06  W/(m  K) 
and  concrete  is  0.8  W/(m  K).®  The  specific  heat  is  the  amount  of  energy  required  to  raise  the 
temperature  of  the  material.  If  the  specific  heat  is  low,  less  energy  is  required  to  raise  the 
temperature.  The  surface  wiU  be  hotter  than  a  material  with  higher  specific  heat  exposed  to  the 
same  energy  (i.e.,  the  sim).  The  specific  heat  of  concrete  is  0.23  cal/fgm  °C),  while  the  specific 
heat  of  asphalt  is  0.40  cal/(gm  °C).  It  takes  about  half  as  much  energy  to  raise  the  temperature  of 
concrete  as  it  does  for  asphalt.  These  are  the  reasons  why  the  contrast  ratio  for  the  cracks  in  the 
concrete  of  Area  E  are  negative  (the  temperature  of  the  bulk  is  hotter  than  the  cracks)  at  1000 
hours  whereas  the  cracks  in  the  asphalt  of  Area  D  are  still  hotter  than  the  bulk  at  1000  hours. 

During  the  hottest  part  of  the  day  the  cracks  are  least  visible,  as  seen  from  the  plots  in 
Figures  5-44  through  5-46.  There  is  another  crossover  point  when  the  surface  begins  to  cool  and 
the  cracks  appear  hotter  than  the  bulk  material.  This  is  because  the  heat  energy  is  radiated  from 
the  hotter  subsurface  through  the  cracks.  Unfortunately,  there  is  little  data  for  these  crossover 
points. 
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Figure  5-43.  Contrast  Ratio  vs.  Temperature  for  Voids  in  Areas  A  and  C 
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Figure  5-44.  Contrast  Ratio  vs.  Temperature  for  Cracks  in  Area  D 
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Figure  5-45.  Contrast  Ratio  vs.  Temperature  for  Cracks  in  Area  E 
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Figure  5.46.  Contrast  Ratio  vs.  Temperature  for  Cracks  in  Area  E  with  Negative  Values  Suppressed 
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CHAPTER  6 
IMAGE  PROCESSING 


Image  processing  provides  a  large  range  of  techniques  which  can  be  useful  in  automating 
the  analysis  of  imagery  to  determine  the  amount  of  damage  to  a  runway.  Several  techniques  are 
described  and  applied  to  the  field  test  imagery.  This  study  demonstrates  the  viability  of  some  of 
these  techniques,  and  is  not  meant  to  be  either  exhaustive  or  a  complete  solution  to  the  problem. 


6.1  MATHEMATICAL  MORPHOLOGY 

Grayscale  morphology  techniques  can  be  very  useful  in  cases  where  one  wishes  to 
identify  pixel  clusters  that  are  dark  compared  to  a  surrounding  relatively  light  background  or  pixel 
clusters  that  are  light  compared  to  a  surrounding  relatively  dark  background.  In  particular,  the 
“top-hat”  transformation  (white  or  black)  combined  with  other  basic  morphological  operations 

can  be  shown  to  be  very  useful  in  these  types  of  situations.^  The  white  top-hat  transformation  is 
defined  as  the  absolute  difference  between  the  image  and  the  “opened”  image,  where  the 
“opening”  is  a  basic  morphological  operation  which  is  geometry  based  and  requires  the  use  of  an 
appropriate  “structuring  element.”  Correspondingly,  the  black  top-hat  transformation  is  defined  as 
the  absolute  difference  between  the  image  and  the  “closed”  image.  Again,  an  appropriate 
geometric  structuring  element  is  required. 

Conceptually,  if  we  view  the  grayscale  image  signal  as  a  2-dimensional  relief  then  the  top- 
hat  transformation  will  extract  the  peaks  or  valleys  that  jut  above  or  below  the  local  surrounding 

terrain.^®  In  particular,  peaks  (valleys)  whose  bases  (mouths)  are  narrower  than  the  width  of  the 
flat  2-dimensional  structuring  element  will  be  highlighted.  For  example,  disks,  lines,  and 
polygons  are  commonly  employed  as  structuring  elements  in  the  operations.  The  idea  is  that  by 
changing  the  size  and  shape  of  the  structuring  element  we  may  control  which  objects  are  extracted 
from  the  image,  based  on  their  size  and  shape.  The  figures  below  show  an  example  of  the 
application  of  these  techniques. 

In  this  example,  the  original  image  shown  in  Figure  6.1  was  opened  using  a  disk  of 
diameter  15  pixels  and  the  result  was  subtracted  from  the  original.  The  top-hat  transformed 
image  is  also  shown  in  Figure  6-1.  Now,  the  image  need  only  be  thresholded  to  extract  the 
desired  signal  peaks  (bottom  image  in  Figure  6-1). 

In  more  complicated  situations,  further  processing  following  the  thresholding  step  may  be 
needed  to  eliminate  groups  of  pixels  that  are  not  the  desired  size  and  shape.  Further,  if  we  are 
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looking  for  pixel  clusters  of  different  sizes  we  may  need  to  perform  these  operations  several  times 
with  different  sized  and  shaped  structuring  elements  and  then  add  the  results  together.  This  can 
best  be  understood  as  a  “sieving  process,”  i.e.,  if  an  image  is  considered  as  a  collection  of  grains, 
then  whether  or  not  an  individual  grain  will  pass  through  the  sieve  depends  on  its  size  and  shape 

relative  to  the  mesh  of  the  sieve.”  A  more  difficult  example  is  shown  below  where  extended 
processing  was  required.  The  original  image  is  shown  in  Figure  6-2  and  the  extracted  areas  of 
interest  are  shown  below.  It  can  be  readily  understood  that  the  false  detections  that  are  difficult  to 
eliminate  are  those  that  are  the  same  size,  shape,  and  relative  brightness  as  the  patches  we  are 
seeking.  In  Figure  6-2  the  false  detections  are  marked  with  circles. 

It  may  also  prove  advantageous  in  these  problems  to  employ  a  “high-boost  filter.”  This 
filter  seeks  out  small  clusters  of  pixels  that  are  slightly  brighter  than  the  immediate  background.  It 
has  been  successfully  employed  in  astronomical  applications  and  has  recently  been  utilized  to 

identify  microcalcifications  in  mammograms. The  results  gained  from  using  the  high-boost 
filter  are  similar  to  those  presented  here  although  at  a  slightly  higher  computational  cost.  One 
advantage  of  using  this  filter  however  is  that  the  need  for  a  geometric  structuring  element  is 
eliminated. 

It  should  be  mentioned  that  although  these  morphological  techniques  are  very  powerful, 
they  do  need  to  be  customized  somewhat  for  each  individual  class  of  images  and  objects  of 
interest.  One  potential  solution  to  this  problem  would  be  to  employ  an  adaptive  white  (or  black) 

top-hat  operator.  These  operators  address  the  problems  that  can  occur  when  one  attempts  to 
choose  a  single  structuring  element  size  in  cases  where  the  objects  of  interest  have  multiple  or 
unknown  widths.  Such  adaptive  methods  warrant  further  study  given  the  variable  nature  of  the 
runway  corruption. 

Although  we  have  discussed  difficult  cases  so  far,  where  the  difference  in  contrast 
between  target  and  background  is  slight,  it  should  be  noted  that  even  simpler  techniques  may  be 
applied  in  cases  where  there  is  significant  contrast. 

As  is  shown  in  Figure  6-3,  we  may  easily  extract  the  light  regions  by  thresholding  the 
image  at  an  appropriate  gray  level.  The  example  bellow  required  using  two  different  thresholds 
and  adding  the  results  together  after  performing  some  simple  morphological  operations. 

Thus,  the  primary  conclusion  of  this  portion  of  the  study  is  the  following:  algorithms 
currently  exist  that  are  capable  of  extracting  interesting  regions  from  an  image  that  have  some 
degree  of  contrast  with  the  local  background.  These  techniques  vary  both  in  complexity  and  in 
their  ability  to  be  implemented  in  a  general  and  automated  context.  Further  investigation  using  a 
large  ground-truthed  database  would  be  needed  in  order  to  develop  a  useful  and  fully  automated 
procedure. 
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Figure  6-1.  Original  (Thermal)  Image  of  Area  A  taken  at  1845,  Top-Hat 
Transformed  Image,  and  Thresholded  Top-Hat  Transformed  Image 
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Figure  6-2.  Original  (Thermal)  Image  of  Area  C  Taken  at  1722 
with  Extracted  Areas  of  Interest.  False  alarms  are  circled. 


Figure  6-3.  Original  (Thermal)  Image  From  Area  I  Taken  at  1931  with  Holes  Extracted 
Through  Thresholding 
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6.2  MAP  SMOOTHING 


Intensity  differences  can  be  used  to  detect  hidden  voids  in  thermal  images,  as  discussed 
above.  It  can  be  useful  to  smooth  these  images  to  better  determine  the  extent  of  the  void.  One 

method  for  performing  this  smoothing  is  via  Maximum  A  Posteriori  (MAP)  estimation.*'^  In  the 
implementation  described  here,  the  MAP  estimator  utilizes  a  mixture  model  which  has  been  fit  to 
the  intensity  values.  The  intensity  values  were  modeled  as  a  mixture  of  two  components,  a  low 
intensity  and  high  intensity  component.  Once  a  model  is  fit,  the  MAP  precedes  as  follows: 

Let 


where 


S 

XE  {low,  high} 


(6-1) 


/(v)  =  X  n/^iv) 

XE  [low,  high} 


(6-2) 


is  the  mixture  model  at  the  pixel  v.  The  method  proceeds  by  replacing  the  posterior  probability  of 
each  pixel  with  that  obtained  by  considering  its  neighbors  as  indicated  below.  For  this  example, 
the  neighbors  consisted  of  a  3x3  neighborhood  around  the  pixel.  This  is  iterated  until 
convergence. 
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aZ-^Cv)  =  (6-5) 

The  new  image  M  is  a  mask  with  nonzero  values  indicating  regions  of  greater  than  normal 
intensity.  An  example  is  shown  in  Figure  6-4. 
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Figure  6-4.  Original  (Thermal)  Image  of  Area  A  Taken  at  1845  and  MAP  Region 


6.3  HOUGH  TRANSFORM 

One  way  to  characterize  certain  types  of  damage  to  runways  is  as  geometric  objects. 
Troughs  and  gouges  tend  to  be  long  and  thin,  with  roughly  parallel  delimitating  lines.  Thus  it 
could  be  useful  to  detect  lines  within  an  image,  for  further  processing  to  determine  whether  the 
lines  are  indicative  of  damage  or  not.  Note  that  lines  can  occur  from  many  benign  processes  as 
well:  tire  tracks  and  painted  lines  in  the  visible  spectrum,  the  edges  of  the  runway  and  the  edges 
of  segments  of  the  runways  in  both  visible  and  thermal  images  can  show  up  as  lines.  Thus  further 
processing  is  required  to  determine  whether  the  structures  indicate  damage  to  the  runway. 

Edge  detection  is  a  well  researched  area  within  image  processing,  and  many  techniques 
have  been  proposed.  Most  work  fairly  well  for  images  with  little  noise  and  fairly  well  defined 
edges.  However,  the  thermal  images  have  a  considerable  amount  of  noise,  and  the  edge  maps 

generated  tend  to  be  very  difficult  to  interpret  (see  Figure  6-5,  which  shows  a  Sobel  edge  map^^). 
Further,  the  edges  are  not  grouped  into  line  segments,  and  it  is  not  clear  how  best  to  accomplish 
this  grouping. 
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Figure  6-5.  Original  (Thermal)  Image  of  Area  A  Taken  at  1845  and  Sobel  Transform 


The  Hough  transform^^  was  designed  to  accomplish  the  grouping  of  edges  into  lines.  In 
the  simplest  implementation,  the  Hough  transform  operates  on  an  edge  map,  obtained  by 
thresholding  an  edge  extraction  algorithm  such  as  the  Sobel  transform.  The  transform  works  by 
finding  neighboring  pixels  which  lie  on  a  line,  and  keeping  track  of  these  through  an  accumulator 
matrix.  In  essence,  for  each  line  defined  in  polar  coordinates  as 

d  =  xcos  (0)  +ysin  (0)  ,  (6-6) 

each  pair  of  neighboring  pixels  which  are  both  on  an  edge  increments  an  accumulator  for  the 
appropriate  line  as  defined  by  d  and  0.  Large  values  in  the  accumulator  indicate  lines. 
Generalizations  of  this  basic  idea  to  gray  scale  images  and  more  general  shapes  are  possible,  and 

have  been  carried  out  by  a  number  of  researchers.^^ 
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Figure  6-6.  Original  (Thermal)  Image  of  Area  M  Taken  at  1832  and  Hough  Transform 

As  can  be  seen  in  the  examples  in  Figures  6-6  and  6-7,  the  Hough  transform  works  quite 
well  on  the  visible  images,  where  the  noise  level  is  fairly  low,  however  it  does  not  work  as  well  on 
most  of  the  thermal  images,  as  a  result  of  the  higher  noise  level.  There  are  some  noise  reduction 
techniques  which  could  be  applied  to  these  images,  and  this  could  be  an  area  of  further 
investigation 


6.4  TEXTURE  FEATURES 

For  some  kinds  of  damage  the  above  algorithms  provide  some  tools  which  can  be  used  for 
detection,  however  these  are  not  necessarily  appropriate  for  other  kinds  of  damage.  For  example, 
bomb  craters  do  not  necessarily  have  linear  sides,  gravel  is  not  necessarily  a  different  temperature 
than  the  runway,  etc.  These  problems  will  require  other  kinds  of  algorithms. 
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Figure  6-7.  Original  (Visual)  Image  of  Area  E  Taken  at  1738,  Sobel  Edge  Detection  for  the 
Image,  and  Hough  Transform 
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One  such  method,  which  may  be  useful  in  detecting  gravel  or  patches  on  the  runway,  is  the 

use  of  texture  features.  A  lot  of  work  has  been  done  in  this  area*^’^^  for  a  number  of  applications, 
and  there  is  reason  to  believe  it  could  be  useful  for  some  of  these  problems  as  well.  For  example, 
gravel  should  have  a  different  texture  than  a  clean  runway,  and  it  should  be  possible  to  detect  it. 
However,  lots  of  other  things  will  have  different  textures  as  well,  for  example  leaves,  dirt,  patches 
to  the  runway,  paint  spills,  etc.,  and  it  is  not  clear  what  texture  features  are  appropriate  to 
distinguish  between  these  different  conditions. 

To  some  degree  the  use  of  thermal  imaging  can  reduce  the  problems  with  some  of  the 
above  conditions,  however  as  has  been  seen  thermal  imagery  is  quite  noisy  and  may  not  detect 
many  of  the  objects  of  interest.  A  thorough  study  of  these  issues  would  be  of  interest,  and  could 
possibly  indicate  both  the  utility  and  the  limitations  of  the  different  imagery  modalities. 

There  are  many  different  texture  measures  which  can  be  used  on  this  imagery  to 
characterize  different  types  of  damage  to  the  runway.  The  large  number  of  possible  features 
requires  some  time  to  process  and  analyze,  and  so  would  require  more  time  than  was  available  for 
this  study.  Once  the  necessary  features  have  been  identified,  they  could  be  used  to  classify  the 
regions  of  the  runway  in  near  real  time. 

An  example  of  one  of  the  texture  features  investigated  is  shown  in  Figure  6-8.  This  is  a 

Laws  texture  feature,  which  is  constructed  by  convolving  the  image  with  a  particular  mask.  It  is 
clear  from  the  figure  that  small  gravel  has  a  different  texture,  under  this  measure,  than  larger  rocks 
or  smooth  runways.  Thus,  by  using  the  appropriate  texture  features,  it  would  be  possible  to 
distinguish  between  different  kinds  of  clutter  on  the  runway.  Similar  algorithms  could  be  used  to 
distinguish  other  types  of  damage  such  as  extensive  cracking  of  the  runway  or  other  types  of 
clutter  on  the  runway. 

With  a  limited  image  set  and  time  for  analysis,  we  could  only  give  some  indications  of  the 
kind  of  algorithms  that  can  be  applied  to  these  images.  The  morphology  algorithms  can  detect 
objects  which  are  larger  or  smaller  than  a  given  size.  Edge  extraction  algorithms  and  Hough 
transforms  can  be  used  to  find  linear  structures.  The  Hough  transform  can  be  generalized  to  other 
shapes  than  lines.  Different  texture  features  can  be  extracted  to  distinguish  between  different 
kinds  of  objects  or  damage.  The  MAP  estimator  can  be  used  not  only  on  the  raw  gray  scale 
images,  but  also  on  the  features  and  even  on  classification  maps  to  smooth  the  results  and 
eliminate  false  alarms  due  to  noise. 

Clearly  no  single  algorithm  will  work  to  extract  all  the  information  of  interest  from  the 
images.  A  series  of  algorithms  would  need  to  be  developed,  each  one  designed  to  find  a  particular 
structure.  These  algorithms  would  then  work  together  to  give  an  overal  classification  of  the 
damage  to  the  runway. 

In  an  operational  system,  one  would  most  likely  use  video  rather  than  still  imagery.  With 
video  mosaicking  techniques,  the  frames  from  the  video  can  be  overlaid  to  produce  a  single  large 
image  of  the  entire  runway.  The  integration  of  multiple  frames  allows  some  denoising  through 
the  reduntant  information  in  the  frames.  It  also  allows  for  improved  classification  by  reducing 
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false  alarms.  The  mosaic  would  give  a  picture  of  the  entire  runway  with  the  locations  of  any 
trouble  areas  in  the  context  of  the  entire  scene.  This  would  allow  decisions  to  be  based  on  where 
the  damage  occurs  on  the  runway,  rather  than  how  much  damage  there  is  overall. 


Figure  6-8.  Original  (Thermal)  Image  of  Area  H  Taken  at  550  and  Laws  Texture  Feature 


The  mosaic  could  also  be  of  value  in  its  own  right.  Given  a  mosaic  showing  the  runway  at 
a  sufficiently  high  resolution,  a  human  operator  could  classify  the  damage,  given  the  training 
necessary  to  interpret  the  non-visible  spectrum  imagery.  Multiple  mosaics  in  different  spectra 
could  be  registered  to  allow  a  multi-spectral  view  of  the  runway.  This  could  greatly  enhance  the 
detection  and  identification  of  damage,  by  utilizing  the  strengths  of  the  different  imagery. 
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CHAPTER  7 
SUMMARY 


The  various  passive  and  active  imaging  systems  have  been  analyzed  for  application  to 
remote  sensing  of  airfields.  The  purpose  is  to  determine  whether  a  plane  can  be  safely  landed. 
The  criteria  of  interest  are;  type  of  material  (including  differences  in  material,  such  as  patches), 
delamination,  cracks,  holes,  spalls,  debris,  surface  roughness  (large  scale,  such  as  trenches)  and 
subsurface  voids.  It  is  apparent  that  a  single  system  will  not  yield  all  the  critical  information.  A 
combination  of  systems  is  necessary  to  detect  all  of  these  defects.  Table  7-1  contains  a  summary 
of  the  systems  that  have  been  reviewed  and  what  type  of  fault  each  can  detect,  assuming  each  has 
been  optimized  as  outlined  in  Section  2.  Polarimetry  has  not  been  included  in  the  list  because  it  is 
not  a  stand-alone  technique.  Rather,  it  is  used  to  enhance  discrimination  capabilities  when 
coupled  with  an  imaging  system. 


Table  7-1 .  Summary  of  System  Type  and  Capability 


Material 

Type 

Delamina¬ 

tion 

Cracks 

Holes 

Spalls 

Debris 

Surface 

Roughness 

Sub¬ 

surface 

Voids 

YES 

NO 

NO’ 

NO’ 

NO’ 

YES 

NO’ 

NO 

YES 

NO 

NO’ 

NO’ 

NO’ 

NO’ 

NO’ 

NO 

Thermal 

YES” 

YES 

YES 

YES 

YES 

YES 

NO^ 

YES 

NO 

NO 

YES 

YES 

YES 

YES 

YES 

NO 

Scanning 

Lidar 

NO 

NO 

YES 

YES 

YES 

YES 

YES 

NO 

NO 

NO 

YES 

YES 

YES 

YES 

YES 

NO 

Visible 

NO 

NO 

YES 

YES 

YES 

YES 

NO^ 

NO 

May  be  detected  in  individual  frames,  but  not  by  design  of  the  technique. 

**  Can  detect  differences  in  materials  or  hardness  of  dirt,  but  cannot  identify  materials  by  spectral  signature  as  in 


the  multispectral  and  hyperspectral  systems. 

^  Can  marginally  detect,  as  seen  in  field  test  data,  but  depth  information  is  not  obtained. 


The  multispectral,  hyperspectral,  and  thermal  systems  can  distinguish  between  material 
types  based  on  the  spectral  signatures  of  the  materials.  If  there  is  dirt  or  weeds  growing  in  a 
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crack  on  a  runway,  for  example,  these  would  be  distinguishable  from  the  “good  pavement” 
background.  These  systems  can  also  detect  oil,  water,  patches  or  other  anomalies  based  on 
different  materials  present  in  the  image.  The  runway  construction  can  be  determined  with  the  two 
spectral  systems;  i.e.,  dirt,  asphalt,  concrete,  type  of  aggregate,  etc. 

The  thermal  imager  has  the  further  capability  of  detecting  surface  faults  such  as  cracks  and 
holes,  and  subsurface  defects.  The  thermal  imaging  system  does  not  provide  depth  information. 
Thus,  a  hot  spot  may  be  caused  by  sand  or  dirt  on  the  surface,  or  it  may  be  caused  by  a  subsurface 
void.  The  only  way  to  tell  is  to  couple  the  thermal  information  with  another  system.  One 
possibility  is  to  combine  a  thermal  system  with  a  standard  video  camera  (or  other  visible  imaging 
system).  If  the  detected  hot  spot  correlates  with  something  in  the  visible  image,  then  the  defect  is 
at  the  surface.  If  there  is  not  a  defect  in  the  visible  image  to  correspond  to  the  hot  spot  in  the 
thermal  image,  then  the  defect  is  subsurface.  Another  possibility  is  to  incorporate  two  thermal 
cameras  in  a  stereoscopic  system.  Depth  information  is  available  from  the  stereoscopic  setup  and 
any  hot  spots  in  the  thermal  image  can  be  pinpointed  at  the  surface  or  below  the  surface.  A 
drawback  to  the  thermal  imaging  system  is  that,  as  shown  by  the  data  from  the  field  test,  the 
surface  defects  are  best  detected  in  the  early  morning  during  periods  of  heating,  while  the 
subsurface  faults  are  more  easily  detected  in  the  evening  during  periods  of  cooling.  A  second 
drawback  is  that  thermal  imaging  is  not  possible  in  rain,  fog,  or  when  there  is  standing  water  on 
the  runway.  Successful  operation  is  dependent  on  local  weather  conditions. 

A  compelling  reason  for  employing  active  imaging  systems  is  for  increased  signal-to-noise 
and  target  detectability  when  imaging  through  dense  media.  These  systems  are  heavily  employed 
for  underwater  imaging  where  the  attenuation  is  appreciable.  For  imaging  through  air,  unless 
there  is  heavy  fog  or  smoke,  the  difficulty  and  expense  in  the  implementation  of  an  active  system 
are  not  justifiable.  The  reason  the  two  active  systems  have  been  evaluated  in  this  study  is  that 
they  yield  depth  information,  which  most  of  the  passive  systems  do  not.  Since  the  passive 
stereoscopic  system  also  imparts  depth  information,  we  do  not  recommend  further  investigations 
into  active  imaging  for  this  application. 
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CHAPTER  8 
RECOMMENDATIONS 


For  application  to  remote  sensing  of  airfields,  the  thermal  imaging  system  yields  the  most 
information.  We  recommend  further  study  of  this  system.  The  thermal  camera  used  in  the  field 
test  was  built  in  the  late  eighties.  Newer  cameras  have  better  spatial  and  temperature  resolution. 
We  recommend  investigations  into  possible  enhancement  by  incorporation  of  polarimetry  into  the 
thermal  imaging  system.  It  would  be  further  illuminating  to  take  data  of  a  single  area  over  an 
entire  24-hour  period  to  fill  in  the  missing  data  for  the  graphs  in  Figures  5-39  through  5-46.  We 
may  then  be  able  to  find  an  optimal  time/temperature  for  detecting  both  surface  and  subsurface 
defects. 


In  a  deployed  system,  we  recommend  coupling  the  thermal  system  to  a  visible  system  to 
(1)  distinguish  between  surface  and  subsurface  defects,  and  (2)  have  some  means  of  evaluating  the 
airstrip  when  inclement  weather  prohibits  use  of  the  thermal  imager.  This  coupled  system  will  not 
give  depth  information,  thus  surface  roughness  information  will  not  be  available.  If  this  data  is 
absolutely  necessary,  we  recommend  some  combination  of  visible  and  thermal  imaging  with  the 
stereoscopic  technique;  stereoscopic  thermal  imaging  with  and  additional  visible  camera;  or 
stereoscopic  visible  imaging  with  an  additional  thermal  camera.  It  may  be  possible  to  design  a 
stereoscopic  imaging  system  in  which  one  of  the  cameras  is  a  visible  CCD  array  and  the  other  is  a 
thermal  array.  The  cameras  would  have  to  be  specially  made  with  the  same  number  and  size  of 
pixels.  Since  the  technology  for  visible  CCD  is  currently  more  advanced  than  for  IR  detectors, 
the  resolution  will  be  limited  by  present  IR  technology. 

It  is  also  advisable  to  perform  a  rigorous  theoretical  analysis  to  determine  whether  the 
runway  material  (asphalt,  concrete,  etc.)  can  be  determined  from  the  thermal  images  based  on 
knowledge  of  the  material  properties  (emissivity,  thermal  conductivity,  and  heat  capacity)  and  the 
ambient  temperature. 
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